Method and device for gesture recognition diagnostics for device orientation

ABSTRACT

Systems, circuits, and devices for recognizing gestures are discussed. A mobile device includes a housing, an orientation sensor, a camera implemented on the housing, a memory for storing a lookup table comprising multiple gestures and corresponding commands, and a controller coupled to the orientation sensor, the camera, and the memory. The controller is configured to generate trace data corresponding to a gesture captured by the camera, wherein x, y, and z coordinates of the trace data are applied according to an orientation of the housing during the gesture. The controller is also configured to determine an orientation angle of the housing detected by the orientation sensor. The controller is further configured to recognize the gesture through accessing the lookup table based on the trace data and the orientation angle of the housing.

BACKGROUND OF THE INVENTION

1. Field of the Invention

Embodiments of the disclosure generally relate to the field ofelectronics, and more particularly to control systems, circuits, anddevices.

2. Discussion of the Background Art

A mobile device is a pocket-sized computing device, typically having adisplay screen with touch input and/or a miniature keyboard. The inputand output of the mobile device are often combined into a touch-screeninterface. The mobile device includes many applications and includes amobile computer, a mobile interne device, a personal digitalassistant/enterprise digital assistant, a calculator, a handheld gameconsole, a portable media player, a digital still camera, a digitalvideo camera, a digital camcorder, a mobile phone, a pager, a personalnavigation device, etc. Some of the applications of the mobile deviceinclude digitizing notes, sending and receiving invoices, assetmanagement, recording signatures, managing parts, scanning barcodes,etc.

Gesture recognition is a technology that makes it possible for a deviceto interpret (human) gestures via mathematical algorithms. Gestures canoriginate from any bodily motion or state but commonly originate fromthe face or hand. Gesture recognition is a way for computers or machinesto understand human body language, thus building a richer bridge betweenmachines and humans than text-based user interfaces or evengraphic-based user interfaces which still limit the majority of input tokeyboard and mouse.

SUMMARY OF THE INVENTION

This summary is provided to comply with 37 C.F.R. §1.73, requiring asummary of the invention briefly indicating the nature and substance ofthe invention. It is submitted with the understanding that it will notbe used to interpret or limit the scope or meaning of the claims.

Systems, circuits, and devices for recognizing gestures are discussed.In one embodiment of the invention, there is a mobile device forrecognizing a gesture includes a housing, an orientation sensor, acamera implemented on the housing, a memory for storing a lookup tablecomprising multiple gestures and corresponding commands, and acontroller coupled to the orientation sensor, the camera, and thememory. The controller is configured to generate trace datacorresponding to a gesture captured by the camera, where x, y, and zcoordinates of the trace data are applied according to an orientation ofthe housing during the gesture. The controller is also configured todetermine an orientation angle of the housing detected by theorientation sensor. The controller is further configured to recognizethe gesture through accessing the lookup table based on the trace dataand the orientation angle of the housing.

In another embodiment of the invention, there is a mobile device forrepositioning a pointer includes a housing, a display unit, anorientation sensor, a camera implemented on the housing, and acontroller coupled to the display unit, the orientation sensor, and thecamera. The controller is configured to generate a pointer on a unitportion of an image displayed on the display unit in response to acapture by the camera of a remote pointing object pointing to the unitportion on the image on the display unit, where the image is displayedon the display unit according to a first orientation. The controller isalso configured to display the image on the display unit according to asecond orientation when an orientation of the housing detected by theorientation sensor indicates a shift to the second orientation. Thecontroller is further configured to identify coordinates of the unitportion of the image displayed according to the second orientation anddisplay the pointer on the unit portion of the image based on thecoordinates of the unit portion of the image displayed according to thesecond orientation.

In yet another embodiment of the invention, there is a mobile device forrepositioning a pointer includes a housing, a display unit, anorientation sensor, a camera implemented on the housing, a controllercoupled to the display unit, the orientation sensor, and the camera. Thecontroller is configured to generate a pointer on a first positionproximate to an image displayed on the display unit in response to acapture by the camera of a remote pointing object pointing to the firstposition on the display unit, where the image is displayed on thedisplay unit according to a first orientation. The controller is alsoconfigured to measure distances between the first position and each oftwo or more unit portions on the image, and display the image on thedisplay unit according to a second orientation when an orientation ofthe housing detected by the orientation sensor indicates a shift to thesecond orientation. The controller is further configured to identifycoordinates of the two or more unit portions of the image displayedaccording to the second orientation and display the pointer on a secondposition proximate to the image based on the coordinates of the two ormore unit portions of the image displayed according to the secondorientation and the distances between the first position and each of thetwo or more unit portions.

In yet another embodiment of the invention, there is an image displaydevice that includes a housing; an orientation sensor; a display; acamera; and a controller. The orientation sensor is configured todetermine an orientation angle of the housing relative to apredetermined coordinate system. The camera is configured to capture auser gesture in a field of view of the camera. The controller isoperatively coupled to the orientation sensor, the camera, and thedisplay, and is configured to convert the gesture to an executioncommand based on the orientation angle.

In yet another embodiment of the invention, there is an image displaydevice that includes a housing; an orientation sensor; a display; acamera; and a controller. The orientation sensor is configured todetermine an orientation angle of the housing relative to apredetermined coordinate system. The camera is configured to capture auser gesture in a field of view of the camera. The controller isoperatively coupled to the orientation sensor, the camera, and thedisplay, and is configured to display a pointer on the display at aspecific location relative to the displayed image, the specific locationcorresponding to a first orientation of the housing and a detectedlocation of the object, and when the orientation of the housing ischanged to a second orientation, reorient the image according to thesecond orientation and while maintaining the pointer at the specificlocation relative to the displayed image.

In yet another embodiment of the invention, there is an image displaydevice that includes a housing; an orientation sensor; a display; acamera; and a controller. The orientation sensor is configured todetermine an orientation angle of the housing relative to apredetermined coordinate system. The camera is configured to capture auser gesture in a field of view of the camera. The controller isoperatively coupled to the orientation sensor, the camera, and thedisplay, and is configured to display a pointer on the display at aspecific location, the specific location corresponding to a firstorientation of the housing and a detected location of the object, andwhen the orientation of the housing is changed to a second orientation,reorient the image according to the second orientation and whilemaintaining the pointer at the specific location. The specific locationis an absolute location on the display that is not changed when theorientation of the housing is changed to the second orientation.

Other features of the embodiments will be apparent from the accompanyingdrawings and from the detailed description that follows.

BRIEF DESCRIPTION OF THE VIEWS OF DRAWINGS

Example embodiments are illustrated by way of example and not limitationin the figures of the accompanying drawings, in which like referencesindicate similar elements and in which:

FIG. 1 is an exemplary block diagram of a mobile device for recognizinga gesture and/or for repositioning a pointer upon a shift in theorientation of the mobile device, according to one embodiment of thepresent invention.

FIGS. 2A and 2B illustrate an exemplary view of the mobile device ofFIG. 1 in a first orientation for recognizing a gesture, according toone embodiment of the present invention.

FIGS. 3A and 3B illustrate an exemplary view of the mobile device ofFIG. 1 in a second orientation for recognizing a gesture, according toone embodiment of the present invention.

FIG. 4 illustrates an exemplary view of the trace data in FIG. 2Acaptured by the mobile device of FIG. 1, according to one embodiment ofthe present invention.

FIG. 5 illustrates an exemplary view of the mobile device of FIG. 1 intwo different orientations displaying an image accordingly, according toone embodiment of the present invention.

FIG. 6 illustrates an exemplary process of the mobile device in FIG. 1for recognizing the gesture, according to one embodiment of the presentinvention.

FIG. 7 illustrates another exemplary process of the mobile device inFIG. 1 for recognizing the gesture, according to one embodiment of thepresent invention.

FIG. 8 illustrates a process flow chart of an exemplary processperformed by the mobile device in FIG. 1 for recognizing a gesture,according to one embodiment of the present invention.

FIGS. 9A-9D illustrate an exemplary view of the mobile device of FIG. 1for repositioning a pointer in response to a shift in the orientation ofthe mobile device, according to one embodiment of the present invention.

FIGS. 10A-10D illustrate an exemplary view of a mobile device in adifferent configuration than the mobile device of FIG. 1 forrepositioning a pointer in response to a shift in the orientation of themobile device, according to one embodiment of the present invention.

FIG. 11 illustrates a process flow chart of an exemplary processperformed by the mobile device in FIG. 1 for repositioning a pointer,according to one embodiment of the present invention.

FIGS. 12A-12E illustrate another exemplary view of the mobile device ofFIG. 1 for repositioning a pointer in response to a shift in theorientation of the mobile device, according to one embodiment of thepresent invention.

FIG. 13 illustrates an exemplary view of the mobile device of FIG. 1 indifferent orientations repositioning a pointer in response to a shift inthe orientation of the mobile device, according to one embodiment of thepresent invention.

FIG. 14 illustrates an exemplary view of a pointer being repositioned onthe display unit in response to a shift in the orientation of the mobiledevice in FIG. 1, according to one embodiment of the present invention.

FIG. 15 illustrates a process flow chart of another exemplary processperformed by the mobile device in FIG. 1 for repositioning a pointer,according to one embodiment of the present invention.

FIG. 16 illustrates an exemplary view of the mobile device of FIG. 1 indifferent orientations maintaining the absolute position of a pointer inresponse to a shift in the orientation of the mobile device, accordingto one embodiment of the present invention.

Other features of the present embodiments will be apparent from theaccompanying drawings and from the detailed description that follows.Further, the drawings described herein are for illustration purposesonly and are not intended to limit the scope of the present disclosurein any way.

DETAILED DESCRIPTION

A method, device and/or system are discussed that recognize a gestureagnostic to device orientation. According to embodiments of the presentinvention, the mobile device (e.g., a tablet PC, a smart phone, etc.)may interpret a gesture applied to the mobile device in the same mannerregardless of the orientation of the mobile terminal (e.g., a portraitorientation or a landscape orientation) during the gesture. As theconventional mobile device can be tilted and display an image inmultiple (e.g., two) different orientations, the same gesture applied tothe conventional mobile device may be interpreted differently dependingon its orientation during the gesture. That is, the gesture applied tothe mobile in the portrait orientation (e.g., the upright position) maybe interpreted different from the same gesture applied to the mobiledevice in the landscape position (e.g., inclined ninety (90) degrees tothe right from the upright position).

In order to address limitations associated with the prior art, accordingto various embodiments of the present invention, the mobile device ofthe present invention is configured to generate trace data by capturinga gesture with a camera of the mobile device, determine an orientationangle or tilt angle of the mobile device using an orientation sensor ofthe mobile device, and recognize the gesture based on the trace data andthe orientation angle, where the trace data and orientation angle may becompared with a set of reference data (e.g., shapes of candidategestures, orientation angles, etc.) stored in a memory of the mobiledevice.

Further, as the mobile device may be tilted freely, a pointer based on agesture (e.g., a finger pointing to the pointer position) may be lostwhen the mobile device shifts its orientation. This is especially sosince the user may not able to accurately reposition the pointer as theimage on the mobile device is displayed in the shifted orientation(e.g., from the portrait orientation to the landscape orientation). Inorder to solve this problem, according to various embodiments of thisdisclosure, the mobile device is configured to generate a pointer on aparticular position of an image displayed on the display unit when thecamera captures a remote pointing object (e.g., a finger) pointing tothe position on the image in a first orientation (e.g., the portraitorientation).

The mobile device then displays the image on the display unit accordingto a second orientation when the mobile device shifts to the secondorientation (e.g., the landscape orientation). As the shift in theorientation of the mobile device is carried out, the particular positionof the image may be flagged or tagged with metadata indicating theposition of the pointer relative to the area of the image displayed onthe mobile device in the first orientation. The mobile device furtheridentifies coordinates of the unit portion of the image displayedaccording to the second orientation, and display the pointer on the unitportion of the image based on the coordinates of the unit portion of theimage displayed according to the second orientation.

Reference will now be made in detail to the embodiments of theinvention, examples of which are illustrated in the accompanyingdrawings. While the invention will be described in conjunction with theembodiments, it will be understood that they are not intended to limitthe invention to these embodiments. On the contrary, the disclosure isintended to cover alternatives, modifications and equivalents, which maybe included within the spirit and scope of the invention. Furthermore,in the detailed description, numerous specific details are set forth inorder to provide a thorough understanding of the present disclosure.However, it will be obvious to one of ordinary skill in the art that thepresent disclosure may be practiced without these specific details. Inother instances, well known methods, procedures, components, andcircuits have not been described in detail as not to unnecessarilyobscure aspects of the present invention.

FIG. 1 is an exemplary block diagram of a mobile device 100 forrecognizing a gesture and/or for repositioning a pointer upon anorientation shift, according to one embodiment of the present invention.In FIG. 1, the mobile device 100 comprises a housing 102, an orientationsensor 104, a camera 106 implemented on or in the housing 102, a memory108 for storing a lookup table comprising multiple gestures andcorresponding commands, and a controller 110 operatively coupled to theorientation sensor 104, the camera 106, and the memory 108. In oneexample implementation, the orientation sensor 104 is an accelerometer,a gyroscope, or a magnetometer. In one example implementation, thecamera 106 is a depth-aware camera, a stereo camera, or a digitalcamera.

In one embodiment of the present invention, which is illustrated in FIG.2A through FIG. 8, the controller 110 is configured to recognize agesture by tracking a target object, such as a hand, a finger, a stylus,etc., moving in a vicinity of the mobile device 100 using the camera 106regardless of the orientation of the mobile device 100.

In one embodiment of the present invention, which is illustrated in FIG.9A through FIG. 11, the controller 110 is configured to reposition apointer on the same portion of an image when the image is displayed in adifferent configuration or orientation according to a shift in theorientation of the mobile device 100.

In one embodiment of the present invention, which is illustrated in FIG.12A through FIG. 15, the controller 110 is configured to reposition apointer on the same position with respect to an image in a vicinity ofthe pointer when the image is displayed in a different configuration ororientation according to a shift in the orientation of the mobile device100.

In one embodiment of the present invention, which is illustrated in FIG.16, the controller 110 is configured to maintain a pointer at the sameposition on the display unit irrespective of a shift in the orientationof the mobile device 100.

FIGS. 2A and 2B illustrate an exemplary view of the mobile device 100 ofFIG. 1 in a first orientation for recognizing a gesture, according toone embodiment of the present invention. In FIG. 2A, the controller 110of FIG. 1 is configured to generate trace data 202 which correspond to agesture forming a rectangle when the gesture is captured by the camera106 as a user 204 of the mobile device 100 moves the user's finger toform the rectangle. During the generation of the trace data 202, x, y,and z coordinates 206 of the trace data 202 are applied according to anorientation of the housing 102 during the gesture. In FIG. 2A, they-coordinate of the trace data 202 is formed along the line crossing thecamera 106 in lengthwise direction (e.g., the North-South direction) ofthe mobile device 100. It is appreciated that although six data pointsP1 through P6 are presented as the trace data 202, there could be moreor less than the six data points forming the trace data 202. That is,the rectangle of the trace data 202 may be a circle, oval, triangle orother regular or irregular polygon or shape. Also, the shape may be acompletely closed shape or may be a partially closed shape.

Before, during or after receiving the trace data 202, the controller 110is configured to determine an orientation angle 208 of the housing 102detected by the orientation sensor 104, where the orientation angle 208is obtained by measuring the angle between a line crossing the camera106 in lengthwise direction of the housing 102 in the orientation takenby the mobile device 100 during the gesture and a straight line crossingthe camera 106 in lengthwise direction of the housing 202 in a defaultorientation 210 (e.g., the portrait orientation where the camera 106 islocated at the top center of the mobile device 100). That is, theorientation angle is obtained by measuring an angle between a straightline crossing the camera in a lengthwise direction of the housing and astraight line crossing the camera in a widthwise direction of thehousing. Thus, if a default orientation in FIG. 2A is a portraitorientation, the orientation angle 208 of zero (0) degrees is obtained.Upon determining the orientation angle 208, the controller 110 isconfigured to recognize the gesture through accessing a lookup tablestored in the memory 108 of the mobile device 100 based on the tracedata 202 and the orientation angle 208 of the housing 102.

FIG. 2B illustrates a table 250 listing exemplary coordinates of thetrace data 202 in FIG. 2A. In the table 250, x-coordinate 252 andy-coordinate 254 of P1, P2, P3, P4, P5, and P6 are listed as (8, 5),(11, 5), (13, 5), (13, 3), (11, 3), and (8, 3), respectively, thusforming a shape of a rectangle with its length extending along thewidthwise direction of the mobile device 100 and its width extendingalong the lengthwise direction of the mobile device 100, and beinginterpreted by the controller 110 as the gesture of forming a rectanglewith the length of the rectangle extending along the x-axis of the x, yand z-coordinates 206.

FIGS. 3A and 3B illustrate an exemplary view of the mobile device 100 ofFIG. 1 in a second orientation for recognizing a gesture, according toone embodiment of the present invention. In FIG. 3A, the controller 110of FIG. 1 is configured to generate trace data 302 which correspond to agesture forming a rectangle when the gesture is captured by the camera106 as the user 204 of the mobile device 100 moves the user's finger toform the rectangle. During the generation of the trace data 302, x, y,and z coordinates 304 of the trace data 302 are applied according to anorientation of the housing 102 during the gesture. In FIG. 3A, they-coordinate of the trace data 302 is formed along the line crossing thecamera 106 in lengthwise direction of the mobile device 100. It isappreciated that although six data points Q1 through Q6 are presented asthe trace data 302, there could be more or less than the six data pointsforming the trace data 302. That is, the rectangle of trace data 302 maybe a circle, oval, triangle or other regular or irregular polygon orshape. Also, the shape may be a completely closed shape or may be apartially closed shape.

Before, during or after receiving the trace data 302, the controller 110is configured to determine an orientation angle 306 of the housing 102detected by the orientation sensor 104, where the orientation angle 306is obtained by measuring the angle between the line crossing the camera106 in lengthwise direction of the housing 102 in the orientation takenby the mobile device 100 during the gesture and a straight line crossingthe camera 106 in the lengthwise direction of the housing 102 in thedefault orientation 210 (e.g., the portrait orientation where the camera106 is located at the top center of the mobile device 100 as in FIG.2A). Thus, in FIG. 3A, the orientation angle 306 of ninety (90) degreesis obtained. Upon determining the orientation angle 306, the controller110 is configured to recognize the gesture through accessing the lookuptable stored in the memory 108 of the mobile device 100 based on thetrace data 302 and the orientation angle 306 of the housing 102.

FIG. 3B illustrates a table 350 listing exemplary coordinates of thetrace data 302 in FIG. 3A. In the table 350, x-coordinate 352 andy-coordinate 354 of Q1, Q2, Q3, Q4, Q5, and Q6 are listed as (9, 9), (9,11), (9, 13), (11, 13), (11, 11), and (11, 9), respectively, thusforming a shape of a rectangle with its length extending along thelengthwise direction of the mobile device 100 and its width extendingalong the widthwise direction of the mobile device 100, and beinginterpreted as a gesture of forming a rectangle with the length of therectangle extending along the y-axis of the x, y, and z-coordinates 304.Therefore, the rectangle or the trace data 302 formed in FIG. 3A isinterpreted different from the rectangle or the trace data 202 formed inFIG. 2A if the interpretation is based solely on the trace data 202 or302 according to the different coordinates (e.g., the x, y,z-coordinates 206 for the mobile device 100 in the portrait orientationand the x, y, z-coordinates 304 for the mobile device 100 in thelandscape orientation) employed by the mobile device in the differentorientations, although the user 204 may use the same gesture to form thetrace data 202 and the trace data 302.

FIG. 4 illustrates an exemplary view of the trace data 202 in FIG. 2Acaptured by the mobile device 100 of FIG. 1, according to one embodimentof the present invention. In FIG. 4, the trace data 202 may be generatedby the camera 106 by capturing multiple frames 402A through 402G from aninitial instance of the gesture to a final instance of the gesture. Themultiple frames 402A through 402G may represent positions of the fingerin the x, y, z-coordinates 206 captured along a time line 404, where P1represents a position of the user's finger at the initial instance ofthe capture by the camera 106 and P7 represents a position of the user'sfinger at the final instance of the capture by the camera 106. It isappreciated that P7 may coincide with the coordinates of P1 to completethe formation of the rectangle.

As the trace data 202 are captured by the camera 106, the x, y, andz-coordinates of the trace data 202 may be determined by the controller110. For example, as illustrated in FIG. 2B, (8, 5), (11, 5), (13, 5),(13, 3), (11, 3), and (8, 3) may be identified as the x, y, and zcoordinates of P1 or P7, P2, P3, P4, P5, and P6, respectively.

FIG. 5 illustrates an exemplary view of the mobile device 100 of FIG. 1in two different orientations displaying an image accordingly, accordingto one embodiment of the present invention. In FIG. 5, the image 212 isdisplayed on the display unit 112 of the mobile device 100 in a portraitorientation so long as the orientation angle of the housing 102 or themobile device 100 is less than a threshold angle 502, whereas image 212is displayed on the display unit 112 in a landscape orientation when theorientation angle of the housing is greater than the threshold angle.

More particularly, in FIG. 5, the orientation angle of the mobile device100 in the first, second, and third instance is less than the thresholdangle 502 (e.g., less than 15 degrees), so the mobile device 100maintains its portrait orientation for each of the three instances.Accordingly, the image 212 is displayed on the display unit 112 in theportrait orientation. Conversely, when the orientation angle of themobile device 100 in the fourth instance is detected by the orientationsensor 104 to be greater than the threshold angle 502, the mobile device100 shifts to the landscape orientation. Accordingly, the image 308 isdisplayed on the display unit 112 in the landscape orientation.

FIG. 6 illustrates an exemplary process of the mobile device 100 in FIG.1 for recognizing the gesture, according to one embodiment of thepresent invention. In step 602, the trace data 302 and the orientationangle of ninety (90) degrees are processed, where the length of theshape of the trace data 302, i.e., the rectangle, is along the y-axis ofthe x, y, and z-coordinates 304. However, in step 604, the controller110 of the mobile device 100 generates a trace data 608 through applyingninety (90) degrees of the orientation angle associated with the tracedata 302.

As a result, the gesture, which generates the trace data 608, i.e., thetrace data 302 applied with the ninety degree orientation angle, areinterpreted by the controller 110 as the gesture which corresponds tothe command for displaying a menu horizontally. The identification ofthe gesture is performed by accessing a lookup table 610 which includemultiple gestures 612 and respective commands 614. Thus, from the lookuptable 610, the trace data 302 and the ninety degree orientation angleare processed to identify the gesture in a shape of a rectangle with itslength ‘c’ extending horizontally, rather than the gesture in a shape ofa rectangle with its length ‘e’ extending vertically.

FIG. 7 illustrates another exemplary process of the mobile device 100 inFIG. 1 for recognizing the gesture, according to one embodiment of thepresent invention. As illustrated in FIG. 1, the memory 108 may includea lookup table 702 which may be accessed to identify the gestureassociated with the trace data 202 or the trace data 302. In FIG. 7, thelookup table 702 comprises multiple combinations of a trace data 704 andan orientation angle 706 and a gesture 708 associated with eachcombination as well as a command 710 which corresponds to the gesture708.

In FIG. 2A, the gesture performed to the mobile device 100 in theportrait orientation results in the trace data 202, i.e., a shape of arectangle with its length ‘a’ extending horizontally, and the zero (0)degree orientation angle. Through accessing the lookup table 702 basedon the trace data 202 and the zero degree orientation angle, the gesturein the shape of a rectangle with its length ‘e’ extending horizontallyis obtained, and the command is executed to ‘display a main menuhorizontally.’

In FIG. 3A, the gesture performed to the mobile device 100 in thelandscape orientation results in the trace data 302, i.e., a shape of arectangle with its length ‘a’ extending vertically, and the ninety (90)degree orientation angle. Through accessing the lookup table 702 basedon the trace data 302 and the ninety degree orientation angle, thegesture in the shape of a rectangle with its length ‘e’ extendinghorizontally is obtained, and the command is executed to ‘display a mainmenu horizontally.’ Thus, the same gesture applied to the mobile device100 in the two different orientations, as in FIG. 2A and FIG. 3A, isinterpreted in the same manner by the mobile device 100 regardless ofits orientation.

FIG. 8 illustrates a process flow chart of an exemplary processperformed by the mobile device 100 in FIG. 1 for recognizing a gesture,according to one embodiment of the present invention. In keeping withthe previous examples, particular components described with respect toFIG. 1 through FIG. 7 are referenced as performing the process in FIG.8. It should be noted that the components that are referred to asperforming the process are for illustrative purposes only. In someimplementations, other components, or a different combination ofcomponents within a system or device, may instead perform the process inFIG. 8.

In operation 802, as illustrated in FIG. 2A and FIG. 3A, the trace data202 or 302 corresponding to a gesture captured by the camera 106 of themobile device 100 are generated.

In one embodiment of the present invention, the image data 212 isdisplayed on the display unit 112 in the portrait orientation when theorientation angle 208 of the housing 102 is less than the thresholdangle, whereas the image data 308 is displayed on the display unit 112in the landscape orientation when the orientation angle 306 of thehousing 102 is greater than the threshold angle. In one embodiment ofthe present invention, the trace data 202 or 302 is generated bycapturing multiple frames 402A through 402G from the initial instance ofthe gesture to the final instance of the gesture using the camera 106and determining the x, y, and z coordinates of the trace data 202 or 302from the frames 402A through 402G.

Then, in operation 804, the orientation angle 208 or 306 of the mobiledevice 100 is detected by the orientation sensor 104 of the mobiledevice 100. In addition, in operation 806, the gesture is recognized(e.g., identified) based on the lookup table 610 or 702 stored in thememory 108 of the mobile device 100. In one embodiment of the presentinvention, a shape of the trace data 202 or 302 is generated byconnecting the trace data 202 or 302 according to temporal data of thetrace data 202 or 302, rotating the shape of the trace data 202 or 302by the orientation angle 208 or 306, and locating the gesture when thereis a match between the shape of the trace data 202 or 302 and one of thegestures in the lookup table 610 or 702. Further, in operation 808, acommand which corresponds to the gesture is executed through accessingthe lookup table 610 or 702.

It is appreciated that the methods disclosed in FIG. 8 may beimplemented in a form of a machine-readable medium embodying a set ofinstructions that, when executed by a machine, cause the machine toperform any of the operations disclosed herein. Also, steps 802 and 804may be reversed or may occur concurrently.

FIGS. 9A-9D illustrate an exemplary view of the mobile device 100 ofFIG. 1 for repositioning a pointer 902 in response to a shift in theorientation of the mobile device 100, according to one embodiment of thepresent invention. In FIGS. 9A-9D, the mobile device 100 comprises thehousing 102, the display unit 112, the orientation sensor 104, thecamera 106 implemented on the housing 102, and the controller 110coupled to the display unit 112, the orientation sensor 104, and thecamera 106.

In FIG. 9A, the pointer 902 is placed on a unit portion 904 of the image212 displayed on the display unit 112 in response to a capture by thecamera 106 of a remote pointing object 906 (e.g., a finger or a stylus)pointing to the unit portion 904 on the image 212 on the display unit112, where the image 212 is displayed on the display unit 112 accordingto a first orientation (e.g., the portrait orientation). In FIG. 9B, theimage 308 is displayed on the display unit 112 according to a secondorientation (e.g., the landscape orientation) when the orientation ofthe housing 102 detected by the orientation sensor 104 indicates a shiftto the second orientation by the mobile device 100.

In one embodiment of the present invention, the unit portion 904 of theimage 212 may be flagged or tagged with metadata immediately prior tothe shift of the orientation of the mobile device 100 from the firstorientation to the second orientation. In one exemplary implementation,the unit portion 904 of the image 212 may be flagged or tagged withmetadata indicating the position of the pointer 902 on the image 212which coincides with the unit portion 904 if any inclination by themobile device 100 beyond a threshold value (e.g., an inclination anglebeyond which indicates the start of an operation shifting theorientation of the mobile device 100, such as five (5) degrees) isdetected by the orientation sensor 104. In one exemplary implementation,the mobile device 100 may not recognize or process any input (e.g., theremote pointing object 906) while the mobile device 100 is in transitionfrom the portrait orientation to the landscape orientation, thusreducing any error which may occur during the transition. For example,an internal accelerometer or other device may generate an input disablesignal upon detecting that the mobile device 100 is in transition fromthe portrait orientation to the landscape orientation.

Once the mobile device 100 completes the transition to the landscapeorientation, the metadata (e.g., the flag, tag, etc.) indicating theposition of the pointer 902, which previously pointed to the unitportion 904 on the image 212, is accessed and located. Then, in FIG. 9C,new coordinates of the unit portion 908 are obtained based on themetadata. Then, in FIG. 9D, the pointer 902 pointing to the same unitportion 904 of the image 212 is repositioned and displayed on the image308. It is appreciated that the size of the image 308 and the proportionof the length and width for the image 212 may differ from the size ofthe image 212 and the proportion of its length and width, respectively.Accordingly, the pointer in FIG. 9D may be repositioned according to themodification of the image 308 from the image 212.

FIGS. 10A-10D illustrate an exemplary view of a mobile device 1002 indifferent configuration than the mobile device 100 of FIG. 1 forrepositioning a pointer 1010 in response to a shift of its orientation,according to one embodiment of the present invention. In FIGS. 10A-10D,the mobile device 1002 comprises a housing 1004, a camera 1006, adisplay unit 1008, an orientation sensor, and a controller coupled tothe display unit 1008, the orientation sensor, and the camera 1006.

In FIG. 10A, the pointer 1010 is placed on a unit portion 1012 of theimage 308 displayed on the display unit 1008 in response to a capture bythe camera 1006 of a remote pointing object 1014 (e.g., a finger)pointing to the unit portion 1012 on the image 308 on the display unit1008, where the image 308 is displayed on the display unit 1008according to a first orientation (e.g., the landscape orientation). Itis appreciated that for the mobile device 1002 with its cameraimplemented at the center of the mobile device 1002 along its length,the landscape orientation is set as the default orientation.

In FIG. 10B, the image 212 is displayed on the display unit 1008according to a second orientation (e.g., the portrait orientation) whenthe orientation of the housing 1004 detected by the orientation sensorindicates a shift to the second orientation by the mobile device 1002.In one embodiment of the present invention, the unit portion 1012 of theimage 308 may be flagged or tagged with metadata immediately prior tothe shift of the orientation of the mobile device 1002 from the firstorientation to the second orientation. In one exemplary implementation,the unit portion 1012 of the image 308 may be flagged or tagged withmetadata indicating the position of the pointer 1010 on the image 308 ifany inclination by the mobile device 1002 beyond a threshold value(e.g., five (5) degrees) is detected by the orientation sensor. In oneexemplary implementation, the mobile device 1002 may not recognize orprocess any input (e.g., the remote pointing object 1014) while themobile device 1002 is in transition from the portrait orientation to thelandscape orientation. For example, an internal accelerometer or otherdevice may generate an input disable signal upon detecting that themobile device 100 is in transition from the portrait orientation to thelandscape orientation.

Once the mobile device 1002 completes the transition to the portraitorientation, the metadata (e.g., the flag, tag, etc.) indicating theposition of the pointer 1010, which previously pointed to the unitportion 1012 on the image 308, is accessed and located. Then, in FIG.10C, new coordinates of the unit portion 1016 are obtained based on themetadata. Then, in FIG. 10D, the pointer 1010 pointing to the same unitportion 1012 of the image 308 is repositioned and displayed on the image212. It is appreciated that the size of the image 308 and the proportionof its length and width may differ from the size of the image 212 andthe proportion of the length and width of the image 212, respectively.Accordingly, the pointer in FIG. 10D may be repositioned according tothe modification of the image 212 from the image 308.

FIG. 11 illustrates a process flow chart of an exemplary processperformed by the mobile device 100 in FIG. 1 for positioning a pointer,according to one embodiment of the present invention. In keeping withthe previous examples, particular components described with respect toFIG. 9A through FIG. 10D are referenced as performing the process inFIG. 11. It should be noted that the components that are referred to asperforming the process are for illustrative purposes only. In someimplementations, other components, or a different combination ofcomponents within a system or device, may instead perform the process inFIG. 11.

In operation 1102, as illustrated in FIG. 9A, the pointer 902 on theunit portion 904 of the image 212 is displayed on the mobile device 100in the first orientation (e.g., the portrait orientation). In operation1102, as illustrated in FIG. 10A, the pointer 1010 on the unit portion1012 of the image 308 is displayed on the mobile device 1002 in thefirst orientation (e.g., the landscape orientation).

Then, in operation 1104, as illustrated in FIG. 9B, the image 308according to the second orientation (i.e., the landscape orientation) isdisplayed when the mobile device 100 shifts to the second orientation.In one embodiment of the present invention, the unit portion 904 of theimage 212 may be flagged or tagged with metadata immediately prior tothe shift of the orientation of the mobile device 100 from the firstorientation to the second orientation. In one exemplary implementation,the unit portion 904 of the image 212 may be flagged or tagged withmetadata indicating the position of the pointer 902 on the image 212 ifany inclination by the mobile device 100 beyond a threshold value (e.g.,five (5) degrees) is detected by the orientation sensor 104. In oneexemplary implementation, the mobile device 100 may not recognize orprocess any input (e.g., the remote pointing object 906) while themobile device 100 is in transition from the portrait orientation to thelandscape orientation.

In operation 1104, as illustrated in FIG. 10B, the image 212 accordingto the second orientation (i.e., the portrait orientation) is displayedwhen the mobile device 1002 shifts to the second orientation. In oneembodiment of the present invention, the unit portion 1012 of the image308 may be flagged or tagged with metadata immediately prior to theshift of the orientation of the mobile device 1002 from the firstorientation to the second orientation. In one exemplary implementation,the unit portion 1012 of the image 308 may be flagged or tagged withmetadata indicating the position of the pointer 1010 on the image 308 ifany inclination by the mobile device 1002 beyond a threshold value(e.g., five (5) degrees) is detected by the orientation sensor 104. Inone exemplary implementation, the mobile device 1002 may not recognizeor process any input (e.g., the remote pointing object 1014) while themobile device 1002 is in transition from the portrait orientation to thelandscape orientation.

In addition, in operation 1106, as illustrated in FIG. 9C, thecoordinates of the unit portion 908 of the image 308 are displayedaccording to the second orientation. In operation 1106, as illustratedin FIG. 10C, the coordinates of the unit portion 1016 of the image 212are displayed according to the second orientation. In operation 1108, asillustrated in FIG. 9D, the pointer 902 is displayed on the unit portion904 of the image 308 based on the coordinates of the unit portion 908 ofthe image 308 displayed according to the second orientation. Inoperation 1106, as illustrated in FIG. 10D, the pointer 1010 isdisplayed on the unit portion 1012 of the image 212 based on thecoordinates of the unit portion 1016 of the image 212 displayedaccording to the second orientation.

It is appreciated that the methods disclosed in FIG. 11 may beimplemented in a form of a machine-readable medium embodying a set ofinstructions that, when executed by a machine, cause the machine toperform any of the operations disclosed herein. Also, steps 1102 and1104 may be reversed or may occur concurrently.

FIGS. 12A-12E illustrate another exemplary view of the mobile device 100of FIG. 1 for repositioning a pointer 1202 in response to a shift in theorientation of the mobile device 100, according to one embodiment of thepresent invention. In FIGS. 12A-12E, the mobile device 100 comprises thehousing 102, the orientation sensor 104, the camera 106 implemented onthe housing 102, the display unit 112, and the controller 110 coupled tothe orientation sensor 104, and the camera 106, and the display unit112.

In FIG. 12A, the pointer 1202 on a first position 1204 proximate to theimage 212 displayed on the display unit 112 is generated in response toa capture by the camera 106 of the remote pointing object 906 pointingto the first position 1204 on the display unit 112, where the image 212is displayed on the display unit 112 according to a first orientation(e.g., the portrait orientation).

In FIG. 12B, a first distance 1208 between the first position 1204 and afirst unit portion 1206 on the image 212 and a second distance 1212between the first position 1204 and a second unit portion 1210 aremeasured. It is appreciated that the process of flagging or taggingmetadata to each of the first unit portion 1206 and the second unitportion 1210 is similar to the process described for flagging or taggingthe metadata to the unit portion 904 in FIG. 9B or 1012 in FIG. 10B. Itis also appreciate that more than two distances may be measured,although the measurements of the two distances are illustrated in FIG.12B. In FIG. 12C, the image 308 is displayed on the display unit 112according to a second orientation (e.g., the landscape orientation) whenthe orientation of the housing 102 detected by the orientation sensor104 indicates a shift to the second orientation by the mobile device100. In one embodiment of the present invention, the first unit portion1206 and the second unit portion 1210 of the image 212 may be flagged ortagged with metadata immediately prior to the shift of the orientationof the mobile device 100 from the first orientation to the secondorientation. In one exemplary implementation, the first unit portion1206 and the second unit portion 1210 of the image 212 may be flagged ortagged with metadata indicating their positions with respect to thewhole area of the image 212 if any inclination by the mobile device 100beyond a threshold value (e.g., five (5) degrees) is detected by theorientation sensor 104. Further, the first distance 1208 and the seconddistance 1212 may be stored in the memory 108 of the mobile device 100.In one exemplary implementation, the mobile device 100 may not recognizeor process any input (e.g., the remote pointing object 906) while themobile device 100 is in transition from the first orientation to thesecond orientation. For example, an internal accelerometer or otherdevice may generate an input disable signal upon detecting that themobile device 100 is in transition from the portrait orientation to thelandscape orientation.

Once the mobile device 100 completes the transition to the landscapeorientation, the metadata (e.g., the flag, tag, etc.) indicating thelocations of the first unit portion 1206 and the second unit portion1210 is accessed and located as well as the first distance 1208 and thesecond distance 1212 from the memory 108. Then, in FIG. 12D, coordinatesof the first unit portion 1214 and coordinates of the second unitportion 1216 on the image 308 are obtained based on the metadata.Further, in FIG. 12E, the pointer 1202 is repositioned and displayed onthe image 212 based on the coordinates of the first unit portion 1214and coordinates of the second unit portion 1216 on the image 308 as wellas the first distance 1208 and the second distance 1212. It isappreciated that the size of the image 308 and the proportion of itslength and width may differ from the size of the image 212 and theproportion of its length and width, respectively. Accordingly, thepointer in FIG. 12E may be repositioned according to the modificationfrom the image 212 to the image 308.

FIG. 13 illustrates an exemplary view of the mobile device 100 of FIG. 1in different orientations repositioning a pointer in response to a shiftin the orientation of the mobile device 100, according to one embodimentof the present invention. In FIG. 13, the image 212 is displayed on thedisplay unit 112 of the mobile device 100 in the portrait orientation solong as the orientation angle of the housing 102 or the mobile device100 is less than the threshold angle 502, whereas image 212 is displayedon the display unit 112 in the landscape orientation when theorientation angle of the housing is greater than the threshold angle502.

More particularly, in FIG. 13, the orientation angle of the mobiledevice 100 in the first, second, and third instance is less than thethreshold angle 502 (e.g., 15 degrees), so the mobile device 100maintains its portrait orientation for each of the three instances.Accordingly, the image 212 is displayed on the display unit 112 in theportrait orientation. Conversely, when the orientation angle of themobile device 100 in the fourth and fifth instance is detected by theorientation sensor 104 to be greater than the threshold angle 502, themobile device 100 shifts to the landscape orientation. Accordingly, theimage 308 is displayed on the display unit 112 in the landscapeorientation. Further, the pointer 1202 is maintained at the samephysical position on the display unit 112 (e.g., the top left corner ofthe mobile device 100) while the mobile device 100 maintains theportrait orientation, but jumps to the new physical position on thedisplay unit 112 (e.g., the bottom left corner of the mobile terminal100) upon the shifting of the mobile device 100 to the landscapeorientation. However, the position of the pointer 1202 is maintained thesame with respect to the image displayed on the display unit 112 (e.g.,the image 212 from the portrait orientation and the image 308 for thelandscape orientation). It is appreciated the details of the operationfor repositioning the pointer 1202 is described in details in FIGS.12A-12E.

FIG. 14 illustrates an exemplary view of the pointer 902 beingrepositioned on the display unit 112 in response to a shift of theorientation of the mobile device 100 in FIG. 1, according to oneembodiment of the present invention. In FIG. 14, the pointer 1202 ismaintained while the mobile device 100 maintains the portraitorientation, but jumps to the new position upon the shifting of themobile device 100 to the landscape orientation, as illustrated in apointer track 1402 in dotted line. It is appreciated the details of theoperation for repositioning the pointer 902 (e.g., or the pointer 1202)is described in details in FIGS. 12A-12E.

FIG. 15 illustrates a process flow chart of another exemplary processperformed by the mobile device 100 in FIG. 1 for repositioning apointer, according to one embodiment of the present invention. Inkeeping with the previous examples, particular components described withrespect to FIG. 12A through FIG. 14 are referenced as performing theprocess in FIG. 15. It should be noted that the components that arereferred to as performing the process are for illustrative purposesonly. In some implementations, other components, or a differentcombination of components within a system or device, may instead performthe process in FIG. 15.

In operation 1502, as illustrated in FIG. 12A, the pointer 1202 on thefirst position 1204 proximate to the image 212 displayed on the displayunit 112 is generated and displayed when the camera 106 captures theremote pointing object 906 pointing to the first position 1204 on thedisplay unit 112, where the image 212 is displayed on the display unit112 in the first orientation (e.g., the portrait orientation). Inoperation 1504, as illustrated in FIG. 12B, the first distance 1208between the first position 1204 and the first unit portion 1206 on theimage 212 and the second distance 1212 between the first position 1204and the second unit portion 1210 are measured.

In operation 1506, as illustrated in FIG. 12C, the image 308 isdisplayed on the display unit 112 according to the second orientation(e.g., the landscape orientation) when the orientation of the housing102 detected by the orientation sensor 104 indicates a shift to thesecond orientation by the mobile device 100. In one embodiment of thepresent invention, the first unit portion 1206 and the second unitportion 1210 of the image 212 may be flagged or tagged with metadataimmediately prior to the shift of the orientation of the mobile device100 from the first orientation to the second orientation. Once themobile device 100 completes the transition to the landscape orientation,the metadata (e.g., the flag, tag, etc.) indicating the locations of thefirst unit portion 1206 and the second unit portion 1210 is accessed andlocated as well as the first distance 1208 and the second distance 1212from the memory 108. Then, in operation 1508, as illustrated in FIG.12D, the coordinates of the first unit portion 1214 and the coordinatesof the second unit portion 1216 on the image 308 are obtained based onthe metadata. Further, in operation 1510, as illustrated in FIG. 12E,the pointer 1202 is repositioned and displayed on the image 212 based onthe coordinates of the first unit portion 1214 and coordinates of thesecond unit portion 1216 on the image 308 as well as the first distance1208 and the second distance 1212.

It is appreciated that the methods disclosed in FIG. 15 may beimplemented in a form of a machine-readable medium embodying a set ofinstructions that, when executed by a machine, cause the machine toperform any of the operations disclosed herein. Also, operation 1504 mayoccur after or concurrent with operation 1508.

FIG. 16 illustrates an exemplary view of the mobile device 100 of FIG. 1in different orientations maintaining the absolute position of a pointerin response to a shift in the orientation of the mobile device 100,according to one embodiment of the present invention. In FIG. 16, theimage 212 is displayed on the display unit 112 of the mobile device 100in the portrait orientation so long as the orientation angle of thehousing 102 or the mobile device 100 is less than the threshold angle502, whereas image 212 is displayed on the display unit 112 in thelandscape orientation when the orientation angle of the housing isgreater than the threshold angle 502. Further, the pointer 1602 ismaintained at its original physical position (e.g., the top left corner)even after the mobile device 100 shift from the portrait orientation tothe landscape orientation.

In various embodiments, the systems, circuits and devices described inFIG. 1 through FIG. 16 may enable recognition of a gesture in the samemanner by a mobile device regardless of the orientation of the mobiledevice. In one example, the mobile device may interpret a gesture in thesame manner whether it be in the portrait orientation or the landscapeorientation by tracing data generated from capturing the gesture and thetilt angle of the mobile device. Further, in another example, the mobiledevice may reposition a pointer to the same position on an imagedisplayed on the mobile device in the first orientation even after themobile device shifts to the second orientation and display the imageaccording to the second orientation. This may be achieved by associatingthe pointer position with the unit portion of the image displayed on themobile device in the first orientation, tracking the position of theunit portion in the image displayed on the mobile device in the secondorientation when the mobile device shifts to the second orientation, andthen repositioning the pointer to the position which corresponds to theunit portion of the image displayed on the mobile device in the secondorientation. Moreover, it is appreciated that conventional controldevices (e.g., a wired and/or wireless mechanical, optical, 3D, tactilemice, etc.), systems, and processes which can carry out limited butsimilar tasks performed by the devices, systems, and methods describedin FIG. 1 through FIG. 16 may be used in parallel. For instance, theconventional control devices, system, and/or processes may perform oneor more tasks when the gesture based control devices, systems, and/or orprocesses illustrated in the present disclosure cannot properly carrythem out for any unforeseeable reasons, and vice versa.

The various devices, modules, analyzers, generators, etc. describedherein may be enabled and operated using hardware circuitry (e.g.,complementary metal-oxide-semiconductor (CMOS) based logic circuitry),firmware, software and/or any combination of hardware, firmware, and/orsoftware (e.g., embodied in a machine readable medium). Further, thevarious electrical structure and methods may be embodied usingtransistors, logic gates, and/or electrical circuits (e.g., applicationspecific integrated circuit (ASIC)). Although the present embodimentshave been described with reference to specific example embodiments, itwill be evident that various modifications and changes may be made tothese embodiments without departing from the broader spirit and scope ofthe various embodiments.

1. An image display device, comprising: a housing; an orientation sensorconfigured to determine an orientation angle of the housing; a displayconfigured to display an image; a camera configured to capture a usergesture in a field of view of the camera; and a controller operativelycoupled to the orientation sensor, the camera, and the display, thecontroller configured to convert the user gesture to an executioncommand based on the orientation angle.
 2. The image display device ofclaim 1, wherein the image display device is a mobile device.
 3. Theimage display device of claim 2, wherein the mobile device includes atransceiver configured to communicate to another device.
 4. The imagedisplay device of claim 1, wherein the controller or the camera isconfigured to generate trace data corresponding to the gesture capturedby the camera.
 5. The image display device of claim 4, wherein the tracedata includes at least two of x, y, and z orthogonal coordinatesrelative to one of the camera and the orientation angle.
 6. The imagedisplay device of claim 5, wherein the controller is configured toconvert the captured user gesture into the trace data by capturing, bythe camera, a plurality of frames from an initial instance of thegesture to a final instance of the gesture; and determining the at leasttwo of the x, y, and z orthogonal coordinates of the trace data from theplurality of frames.
 7. The image display device of claim 5, wherein thecontroller is configured to recognize the user gesture based on thetrace data.
 8. The image display device of claim 7, further comprising:a memory operatively coupled to the controller and configured to storegesture information and a gesture-specific execution command thatcorresponds to the orientation angle, wherein the controller isconfigured to access the gesture information and convert the recognizeduser gesture to the gesture-specific execution command that correspondsto the orientation angle.
 9. The image display device of claim 8,wherein the controller is configured to convert the recognized usergesture into the gesture-specific execution command by determining thata pattern of the trace data substantially matches a pattern in thememory.
 10. The image display device of claim 1, wherein the orientationsensor is an accelerometer, a gyroscope, or a magnetometer.
 11. Theimage display device of claim 1, wherein the camera is a depth-awarecamera, a stereo camera, or a digital camera.
 12. The image displaydevice of claim 1, wherein the controller is configured to control thedisplay to display the image in one of a landscape orientation and aportrait orientation when the orientation angle of the housing is lessthan a threshold angle and to display the image in another of thelandscape orientation and the portrait orientation when the orientationangle of the housing is greater than the threshold angle.
 13. The imagedisplay device of claim 1, wherein the controller is configured toobtain the orientation angle by measuring an angle between a straightline crossing the camera in a lengthwise direction of the housing and astraight line crossing the camera in a widthwise direction of thehousing.
 14. The image display device of claim 1, wherein the controlleris further configured to execute the execution command.
 15. An imagedisplay device, comprising: a housing; an orientation sensor configuredto determine an orientation of the housing relative to a predeterminedcoordinate system; a display configured to display an image; a cameraconfigured to capture an object in a field of view of the camera; and acontroller coupled to the display, the orientation sensor and thecamera, the controller configured to: display a pointer on the displayat a specific location relative to the displayed image, the specificlocation corresponding to a first orientation of the housing and adetected location of the object, and when the orientation of the housingis changed to a second orientation, reorient the image according to thesecond orientation while maintaining the pointer at the specificlocation relative to the displayed image.
 16. The image display deviceof claim 15, wherein the controller is configured to identifycoordinates of the pointer when the housing is in the first orientation,and wherein the controller is configured to maintain the pointer at thespecific location relative to the displayed image when the housing is inthe second orientation based on the coordinates.
 17. The image displaydevice of claim 15, wherein the controller is configured to determine afirst distance between the pointer and a first part of the image and asecond distance between the pointer and a second part of the image whenthe housing is in the first orientation, and wherein the controller isconfigured to maintain the pointer at the specific location relative tothe displayed image when the housing is in the second orientation basedon the first and second distances.
 18. The image display device of claim15, wherein the first orientation is one of a landscape orientation anda portrait orientation, and the second orientation is another of thelandscape orientation and the portrait orientation
 19. The image displaydevice of claim 15, wherein the controller is configured to detect thechange of the housing to the second orientation when tilt data generatedby the orientation sensor is greater than a threshold angle.
 20. Animage display device, comprising: a housing; an orientation sensorconfigured to determine an orientation of the housing; a displayconfigured to display an image; a camera configured to capture an objectin a field of view of the camera; and a controller coupled to thedisplay, the orientation sensor and the camera, the controllerconfigured to display a pointer on the display at a specific location,the specific location corresponding to the housing and a detectedlocation of the object, reorient the image according to a secondorientation when the orientation of the housing is changed to the secondorientation, and display a pointer on the display at the specificlocation, the specific location corresponding to the housing and adetected location of the object regardless of the orientation of thehousing.